CMNN: Coupled Modular Neural Network

In this paper, we propose a multi-branch neural network architecture named Coupled Modular Neural Network (CMNN). A CMNN is consisting of ? closely coupled sub-networks, where termed as the branching factor in paper. We call whole super-graph and each sub-network sub-graph. Each sub-graph stand-alone shares common block with other sub-graphs. To effectively leverage simple but easy-to-implement Round-Robin-based learning algorithm. training iteration contains two phases. first phase, choose Round-Robin fashion train it using knowledge (distillation). second fine-tune based on updated This algorithm produces different copy at which acts an improved teacher for sub-graph; one sub-graphs functions new building super-graph. validate test proposed algorithm, conduct experiments CIFAR-10, CIFAR-100, Tiny ImageNet private On-Road-Risk (ORR) datasets. Empirical results all these four datasets indicate that not only obtain strong network, framework can also produce ensemble performance substantiates diversity introduced throughout framework.